I had been pondering to do an end-of-year bug stat post. Niels Thykier forced my hand, so here goes :) The UDD bugs interface currently knows about the following release critical bugs:

• In Total: 1353
  • Affecting Jessie: 476 That's the number we need to get down to zero before the release. They can be split in two big categories:
    • Affecting Jessie and unstable: 410 Those need someone to find a fix, or to finish the work to upload a fix to unstable:
      • 52 bugs are tagged 'patch'. Please help by reviewing the patches, and (if you are a DD) by uploading them.
      • 28 bugs are marked as done, but still affect unstable. This can happen due to missing builds on some architectures, for example. Help investigate!
      • 330 bugs are neither tagged patch, nor marked done. Help make a first step towards resolution!
    • Affecting Jessie only: 66 Those are already fixed in unstable, but the fix still needs to migrate to Jessie. You can help by submitting unblock requests for fixed packages, by investigating why packages do not migrate, or by reviewing submitted unblock requests.
      • 0 bugs are in packages that are unblocked by the release team.
      • 66 bugs are in packages that are not unblocked.

Graphical overview of bug stats thanks to azhag:

I am very pleased to say that RC bug counter for Jessie finally dropped below 500 RC bugs. For comparison, we broke the 700 RC bug curve in the start of November, so that is about 200 RC bugs in about 7-8 weeks (or about 25-28 RC bugs per week). Today, we have about 162 RC bugs on the auto-removal list. Though, I suspect many of the affected packages have reverse dependencies, so their removal from testing may be up 30 days away. Nevertheless, by this time in January, we could be looking at no more than 350 RC bugs left for Jessie.

Yesterday, I have managed to finish up an old pet peeve of mine. I wrote a series of patches to have harness reprocess all packages, which were last checked by an older version of Lintian than the current. It is not completed to the level I would like, but it is good enough for now. The implementation works by processing up to 1024 groups after each daily run. It also have a timer to skip this, if the run has taken more than 4 hours. Packages to be reprocessed are sorted by the version they were last processed should a new version of Lintian be upload before all packages have been reprocessed, the oldest results are refreshed first. A rough estimate suggests than in about 3-4 weeks, the entire archive will have been reprocessed with Lintian 2.5.20. It is by no means faster than a full run (which is about a week), but it is automatic and removes the human factor in keeping Lintian results up to date.

I have decided to step down as main maintainer of Lintian and pass the baton to Bastien Roucari s. This is actually fairly old news, since I announced this almost a month ago. However, I was not very vocal about it until now (so do not be surprised if you had not heard of this before). In the past month, I never once regretted having stepped down. If anything, I should probably have done it a bit earlier. Beyond the lack of motivation, I also realised that I had become an all talk and no do -maintainer. The kind that I have been advocating against publicly. This may sound weird to a lot of people, who knows me as the Lintian guy or Mr. Lintian (or whatever Lintian-related nickname people gave me). But the fact is that Bastien has done more for Lintian in the past month than I have in the past two. Despite stepping down as main developer, I have not left Lintian completely. I am still around for helping/explaining, uploading and keeping lintian.debian.org running.

It looks like we have dropped (or will drop) below 700 RC bugs affecting Jessie today[1]. On a related note, I got my eye on libfso-glib + fso-gsmd, samba + -samba4, mono and haskell. By my count, each of the four groups will (once they migrate) bring at least 4 RC bug fixes with them to testing. Besides that, we still have ~97 RC bugs in the autoremoval queue[2], so we could be looking at 600 RC bugs in a week or two. Please keep up the steady flow of RC bug fixes; but also, please make sure your packages will actually migrate. If you need help interpreting the excuses page, do not hesistate to ask I will not bite. If you want to help, please consider having a look at the cleaner or the Release Team -view on the UDD bug tracker. Sadly not everything there contains actionable problems[3]. [1] http://bugs.debian.org/release-critical/ says 700 while the IRC bot in #debian-devel-changes says we are at 692. [2] http://udd.debian.org/bugs.cgi?release=jessie_and_sid&merged=ign&autoremovals=only&fnewerval=7&flastmodval=7&rc=1&sortby=id&sorto=asc dd-list-like: http://udd.debian.org/cgi-bin/autoremovals.cgi [3] E.g. the kfreebsd-8 bugs appear because kfreebsd-8 is listed in Built-Using for d-i, but the package has already been removed from testing. So there is nothing to do with that (except possibly waiting for a new version of d-i).

Early this month, the first batch of packages were automatically removed from testing due to RC bugs. Personally, I find that the results stand out pretty well on the RC bug graph[1]. There has been a very rapid decline in the number of RC bugs affecting testing a week or two ago, we had over 1000 RC bugs affecting testing and now it has dropped to something like 760. There has also been quite a few RC bug fixes in this time period. I spent a bit of time finding and aging some of the RC bug fixes in sid, so they would migrate to testing faster[2]. On the first batch, we saw something like 40 RC bugs disappear and we still have more RC bugs fixes in the sid->testing pipeline. While looking for packages I could age, I also noticed that we have quite a few RC bug fixes blocked by out of date binaries (which is usually either build issues or uncoordinated transitions). I have been filing a couple of RC bugs for these, but it does not scale do this kind of work manually. If you have uploaded a package recently, please take the 5 minutes to check that it actually built and it has no out of date binaries issues in its excuses. [1] http://bugs.debian.org/release-critical/ [2] The aging was equivalent to an urgency=medium, so reduced the requirement from 10 days to 5 days.

since lintian gained support for vendor profiles, or at least since it has a nice documentation about writing checks & even a tutorial (the latter only local/in the package), I thought about writing some checks in a specific 'pkg-perl' profile. after talking to niels at debconf13, I tried in late august, & realized that at least simple checks are fun to write & really not difficult; luckily niels also reviewed my first attempts. & axel soon joined the fun & added some more checks. so here we are: the new pkg-perl-tools package also ships a lintian vendor profile 'pkg-perl' with a couple of simple but still useful checks that help to enforce our group policy. if you want to use them, install the package and check /usr/share/doc/pkg-perl-tools/README.lintian, if you want to look at the checks, you can also clone the git repo or inspect it in gitweb (directory lintian).

Today, I released Lintian 2.5.19. It is mostly a bug-fix release, but it also features a new tag. The new tag, homepage-in-binary-package, is emitted for packages where the source package has no Homepage-field, but one of its binaries do. The problem is that some tools (e.g. the PTS) only processes the data from the .dsc files (or Sources indices from the mirrors) and would in these cases miss the Homepage of the page. We now also carry a work around for a bug in tool, t1disasm, where it ends up in an infinite recursion (i.e. until the C call stack overflows). That issue is filed as #724571. The work around is to prevent Lintian from exiting with a failure (which causes packages to be auto-rejected when uploaded to Debian archive). These problems are currently reduced to a warning on stderr. In other (Lintian related news), the full run was completed yesterday after only 4 days (rather than the 8, I expected). While that is a pleasant surprise, I cannot find any changes in Lintian since the last run that would explain this difference. My current conjecture is that our last pseudo full run (when we added amd64) included more processing since the amd64 packages would not have any pre-collected information. With the follow up run to catch packages uploaded after the full run, Lintian spent a little more than 3 and half days (~86.5 hours) processing about 20700 package groups. This gives an average of about 240 groups an hour (or ~4 a minute). Though, remember these numbers include some data having been pre-collected from previous run on the majority of packages.

Today I had to pleasure of releasing Lintian 2.5.18.1. It is a minor bug fix release on top of Lintian 2.5.18 (which I also released today ). When I was about to deploy the new version of Lintian on lintian.debian.org, I noticed a couple of errors in the log file that smelled of bugs. Indeed, when I investigated them I found two bugs, which were both introduced in Lintian 2.5.17.

Due to another bug in 2.5.17 and the some of the false-positives fixes, we are now doing a full run on lintian.debian.org. After having run for 7 hours, Lintian passed cipux . My rough estimate suggests that it will take 8 days at the current rate. For comparision, when we added amd64, Lintian spent 5 days processing all the new binaries (plus their source and the old binaries built from those source packages).

Of course estimate is rather rough. In particular, I am not sure that the heavy duty groups (e.g. linux and libreoffice) are evenly spread out among the letters of the alphabeth. Anyhow, there will be no updates on lintian.debian.org for the next week or so.

I have just uploaded Lintian 2.5.18 to unstable. While fixing 22 bugs, it only features 5 new tags.

• debian-changelog-has-wrong-weekday
• debian-rules-missing-good-practice-target-dfsg
• empty-udeb-package
• file-name-in-PATH-is-not-ASCII
• misplaced-extra-member-in-deb

The release also include fixes to some false-positives, such as python:any dependencies triggering python-script-but-no-python-dep, a rewritten README file. We also included a patch to make Lintian accept the Original-Maintainer field by default for non-Debian vendors (even if they do not have a profile and Lintian ends up loading the debian/main profile). We also added support for running Lintian directly from a git checkout or source tree without setting LINTIAN_ROOT (or passing root). Since that was the primary use-case for root that option has now been deprecated. I also had lintian and lintian-info require the include-dir and [no-]user-dir options as the first if given at all. I would like to thank Bastien Roucari s, Gaudenz Steinlin, Gunnar Wolf, J r my Bobbio and Lucas Nussbaum for contributing to Lintian and the many who submitted reports or suggestions for improvements. I would also like to thank Brian hugmeir Fraser, who assisted me in identifying and working around a bug in Perl s glob function when run under threads (filed upstream as RT#119897).

A couple of days ago, the upstream maintainer of autodie, Paul Jamieson Fenwick (PJF), released autodie 2.21. It includes all of the performance improvements I had proposed in Optimizing autodie (plus a few extra ones authored after I wrote the post) and a small fix to one of the tests (due to changes in Carp). As mentioned in my earlier post, these improvements gives about a factor 4 load time improvement in most cases. For the sake of it, I have decided to use Lintian as an example of how good this improvement is. First off, Lintian loads uses autodie in about 75 unique files in the current git master branch.

$ grep -lr 'use autodie' lib/ checks/ collection/ frontend/lintian   wc -l
75

As a simple benchmark, lets consider the following Perl one-liner:

$ /usr/bin/time -f "%es %Us %Ss" perl -Mautodie \
  -e 'print $autodie::VERSION, " => ";' \
  -e 'eval join("\n", map  "package X$_; use autodie;"  (1..74));'

A quick-n-dirty comparison gives:

2.20 => 0.91s 0.90s 0.01s
2.21 => 0.15s 0.15s 0.00s
# NB: the times are "wall, user and sys" in that order, in case you
# are too lazy to read time(1).   

So, 2.21 is now closer to a factor 6 rather than the original factor 4. I have to admit that the timings seem to be very machine dependent as my laptop shows a factor 7 improvement (from ~1.39 to ~0.22s). With at least 370 tests cases running in the Lintian testsuite, I can look forward to a reduction in user time on about 4 minutes.

In my previous post, I explained about the age excuse in Britney. In this post, I will cover the slightly more tricky out of date excuse. A simple example of excuse is:

out of date on mipsel: blinken (from 4:4.8.4-1)

In this simple case, Britney simply states that the binary blinken on mipsel has not yet been rebuilt. Generally, the presence of an excuse like this is sufficient to stop testing migration. The exception to this rule is if the architecture has been marked as not keeping up . At the moment we do not have any such architectures. The tricky part of an out of date -excuse is that Britney simply identifies a symptom and not a cause. In the excuse above, it is not possible to determine if the build failed on that architecture or simply has not finished yet. But the part that probably confuses the most, is when Britney lists binaries no longer built from that source:

out of date on i386: libfsoframework0 (from 0.11.0-1.1)

This case actually happens quite often and is (usually) a sign of a transition has been started. The part that confuses many is that the build-state of their package will be Installed (as in It was built successfully and has been uploaded to the archive ). From there they tend to conclude Britney is broken , where The excuse information is a bit unhelpful might be more accurate. Caveats:

1. If the source package has not yet been built on that architecture, these old binaries are interleaved into the regular list of out of date -binaries.
2. The list of excuses are updated twice a day (and there is also a delay between its update and the PTS picking it up). Thus, if the build-state of your package for that architecture has been Installed for less than 24 hours, you may just want double check before jumping to conclusions.

The problem is that the binary still has reverse dependencies in unstable, so it cannot migrate to testing. When the last reverse dependency has been updated, the FTP masters will usually semi-automatically decruft your package and the excuse will disappear. On the other hand, if the binary still has reverse dependencies, you probably started a transition. In that case, your package will be stuck in unstable until all the reverse dependencies are ready (or, at least, rebuilt to use the replacement package). Generally, it is good if the Release Team was involved before you did this (see this page ). In summary, if your package has Out of date on [architecture] in its excuses it can be one (or more) of:

• Your package is not built yet on the architecture. This is a transient problem, wait for the build servers to build your package.
• Your package FTBFS on that architecture. This is usually a permanent problem where you need to investigate why it happened and how to solve it. Sadly, often it can take weeks before someone files a bug against your package for this build failure, so being proactive might be worth your time.
• The binary is no longer built binary. This is usually a transition. If the binary packages listed has no reverse dependencies in sid and testing, then just wait for the FTP masters to decruft your package.

Every package maintainer has probably looked at the Testing migration section of their package s PTS page and wondered: Okay? What the hell does that mean? . Or perhaps Is that something I should do something about? . Lets have a look at an example excuse (from the PTS)[1]:

excuses:
*  Maintainer: Debian Python Modules Team
*  Too young, only 8 of 10 days old
*  Updating python-scipy fixes old bugs: #691254, #707315
*  Not considered

The first thing to look for is the phrase Valid candidate . If that appears, Britney will attempt to migrate your package to testing. Note the migration may fail, but we can come back to that (in a) later (post). On the other hand, if Not considered appears, then Britney believes your package is not ready to migrate yet. At this point, the excuse will contain 1 or more reasons explaining why Britney thinks your package should not migrate. The most common problem is probably age . In the example above, that is indeed the problem denoted by the Too young, only 8 of 10 days old . Age is incremented on the evening run (i.e. the 10 pm UTC run). The excuse of that run will include the up to date age. So when you see:

  Too young, only 9 of 10 days old

It means that the age criteria will be satisfied in the next evening run . If the age requirement is satisfied, the entry will look like:

    11 days old (needed 10 days)

If there is a Not considered , it means there is something else you need to fix to make your package migrate. Note that the age is based on the day the package was uploaded to sid. If the package migrates to testing and is later removed from testing again, the age will usually be ridiculously high. This is because all the days spent the package spent in testing is also included in the age. [1] It looks similar on the real excuses page, except the excuses: header is actually the name of the source. Is also includes information about the versions involved (i.e. [from-version] to [to-version] ).

We finally managed to deal with one of the major runtime bottlenecks in Lintian. Previously, our usage of file(1) would sometimes spend a long time classifying various text files. In Lintian 2.5.14, we applied a series of patches that allowed us to disable file(1) s ascii test. This means that Lintian will now see all text files as data , which is somewhat uninformative. However, it turns

N: [...] file-info for source:linux/3.2.20-1 done (120.350s)

into

N: [...] file-info for source:linux/3.2.20-1 done (20.984s)

In the same version, we also had Lintian create fewer empty files and directories in its laboratory. For the normal user, this change has hardly any practical effect. But for lintian.d.o, this reduces the minimum inodes consumed for binary entries with 8 or so. For Lintian 2.5.15, we have may end up reducing the memory requirements of Lintian a bit. So far, we got two changes applied that is taking out at least 12MB of RAM usage on the linux source package and 21 of its binaries. Furthermore, Lintian will (if Devel::Size is available) be able to tell how much memory it is using for its caches when run with -dddd .

After converting Lintian to using autodie, I was a bit trouble by the effects it had on Lintian s (start-up) performance. This overhead accumulated to at least a 1s when checking packages. Indeed, I am not the first to complain about this problem. For those who have not heard of it, autodie is a Perl pragma that basically alters subs and built-ins from a check return value for errors -sub (e.g. like C) to throws exception on error -sub (e.g. like Python). So, you can replace open( ) or die with open( ) when autodie is in effect. autodie achieves this by exporting a little sub that wraps the target sub (or built-in). This little wrapper is compiled[1] into the importing package. The problem with this solution is that it ends up compiling these subs once per importing package. Secondly, all of these wrappers are wrapped in something called a leak guard . As autodie is a lexical pragma it is not permitted to leak across file boundaries. The leak guards take care of that by doing a runtime check of the caller and reverting to the original sub/built-in if a leak occurred. These leak guards were also compiled into the importing package. It is probably not much of a surprise to anyone that autodie spent quite a bit of its runtime compiling these wrappers and their leak guards. Together with upstream, I have addressed some of these problems. The first optimization was to allow some wrappers for built-ins to be reused that does not take glob or bareword arguments[2]. Secondly, we started using closures to generate the leak guards. Combined these two effectively cut the load time in half for subsequent loads of autodie, though it did have a negative effect on the call overhead for these wrappers (~1s for 1M calls). That landed in autodie v2.18. The extra runtime overhead of v2.18 was basically caused by the leak guard being split into two (a small closure, which called the real leak guard with an extra argument). We managed to merge these two parts into one without (too much) loss of code maintainability. With that, the call overhead of these wrappers dropped by ~3s (for 1M calls) and thus the wrappers in autodie v2.20 were even faster than the ones in v2.17. Beyond these improvements, I have sent upstream several patches (still pending review) that will improve the performance of autodie even further. If merged, we are looking at another 1.5s (per 1M calls) removed and almost a factor 4 reduction on the load time of autodie. This factor 4 comes from lazily compiling the wrapper on demand, so some of the price is moved to the call overhead. That said, it is a net gain since it is highly unlikely that every module using autodie will use every single wrapped built-in (how often do you use both read and sysread in the same module)? [1] It is compiled via a call to eval using some template code. [2] For the rest of the built-ins, it could probably just use Symbol::qualify to qualify bareword filehandles. Perhaps that will come in a future version of autodie.

During the Wheezy freeze, the Debian release team deployed 3254 hints[1]. This number may include some duplicates (i.e. where two members of the team hinted the same package), it certainly does not include a lot of rejected requests <insert more disclaimers here>. The top hinter was *drum roll* Adam, who did 1799 hints(That is 55% of all hints during the freeze). For comparison, the second and third runner ups added together did 1023 hints (or 31.4%). Put in a different way, on average Julien Cristau and I would both add about 1.5 hints each day and Adam would on his own add 5.6 hints a day. Of course, this is not intended to diminish the work other the rest of the team. Reviewing and unblocking packages is not all there is to a release. Particularly, a great thanks to Cyril Brulebois for his hard work on the Debian Installer (without which Debian Wheezy could not be released at all). Enjoy! [1] Determined by:

  egrep -c 'done 201(3 2-?(07 08 09 10 11 12)) $HINT_FILE

It does not count hints, but the little header we add above our hints. One header can apply to multiple hints (which is good, because udeb unblocks are usually paired with a regular unblock and we don t want to count them as two separate hints).

About 5-6 weeks ago, I wrote about the Wheezy release progress, so it is about time for another update. According to UDD, we are down to 204 RC bugs (down from 249, since my last post). It is not quite the 2.4 RC bugs per day actually it is about 1.1 RC bug a day. Unfortunately, we do not appear to be fixing bugs faster than we are reporting them at the moment. If you look at Richard Hartmann s post from last week, then we had 206 RC bugs left. Even worse, we appear to have regressed between week 7 and 8 of this year (194 to 206). If you want the Wheezy release to happen soon, please consider helping us by providing bug fixes that comply with the Wheezy freeze policy. If the pace of RC bug fixes do not pick up, the alternative is that the release team deals with the bugs. Note that deals generally falls into one of 2 categories. Either we defer/ignore the problem for Wheezy[1] or we remove affected packages from Wheezy/testing. Particularly, if we have to remove packages, they may take reverse dependencies with them as collateral damage. I do not like these tools anymore than you do. But if the RC bugs fixes are not coming in, it is the only two tools we have left. [1] Meaning that at best the bug fix will occur at Wheezy point release at worst, not at all.

In January, I did a TV-shop ad -style post on a little script called lintian-overrider and it prompted Simon to ask:

That s a great tool, but don t you fear it makes unjustified overrides too easy ?

In my experience, people sometimes have issues writing overrides (justified or not). In fact it sometimes leaves them really frustrated with Lintian. If that frustration eventually leads to them reject Lintian, then we are doing the project a disservice. I will quote Russ Allbery as I believe he covered when he wrote:

I care most about all of the regular Debian developers [...] continuing to use Lintian, so that Lintian can stay as effective as it is now at getting people to make archive-wide changes. [...] This only works if we can get nearly everyone uploading packages to run Lintian all the time.

If a handful of people adds overrides for tags they should not have, then we can fix that (e.g. by filing a bug against their packages). But it is unlikely that we will ever make them run Lintian again once they have boycott it.

In December, I wrote a post on the progress of the Wheezy release. Back then, we had 348 RC bugs in testing and today there are about 249 left. A bit of simple math put us at 99 RC bugs fixed since last and an average of about 2.4 RC bugs fixed per day (up from 1.8). Assuming a constant rate, we would be able to release in about 100 days or 3 months (plus 1 week or two) from now. If you want the release earlier than that, fix RC bugs even faster! Though, the data we are looking might be inaccurate. We filed a bug against UDD, which claims that #639407 affects Wheezy while it is in fact fixed as far as the BTS is concerned. If UDD is only wrong in this direction, we can hope for a positive surprise when the UDD bug is fixed. On the other hand, we may also be unpleasently surprised if not. While talking about the UDD, I would like to mention a new feature its bug search script. It is now possible to filter based on whether or not a package is unblocked. If you take this into account (and assume that all unblocked packages will migrate in a timely manner), our RC bug count drops to about 219. It also gives us a much better view of how many packages that could actually use an unblock. which is now down to about 61. In case you have been wondering what is up with the removal of RC buggy leaf packages . We have still been looking at those, but for the last couple of times there have been at most one candidate for a given run (obviously not the same package each time). It felt a bit excessive to send an email to debian-devel for just one RC bug.

Have you ever tried to add a Lintian override only to get it wrong? Fret not, with the Lintian-overrider 2000 such are problems of the past! Simply feed the tag emitted by Lintian to the Lintian-overrider 2000 and it will show you the correct format for the override plus the file to put said overide in. Furthermore, it will show you variants that you may (or may not) want to use instead.

$ echo "W: login: setuid-binary bin/su 4755 root/root"   \
     lintian-overrider --alternative-forms
  --8<-- debian/login.lintian-overrides --8<--
# If you want to override all (present and future) variants
# of this tag, use:
#  setuid-binary
setuid-binary bin/su 4755 root/root
# Alternative forms...
#   login: setuid-binary bin/su 4755 root/root
#   login binary: setuid-binary bin/su 4755 root/root
# For architecture specific overrides, use one of:
#   login [i386-any amd64-any other-archs] binary: setuid-binary bin/su 4755 root/root
#   login [!i386-any !amd64-any !other-archs] binary: setuid-binary bin/su 4755 root/root
  --8<-- End of debian/login.lintian-overrides --8<--

No more fiddling with that stupid syntax. Just feed it to the Lintian-overrider 2000 and instantly you will get the overrides you want! If you sometimes make a copy-waste mistake or just feel life is too short manually update those files, the Lintian-overrider 2000 is the tool for you! With its --source-dir command line option, your lintian overrrides are updated automatically! The Lintian-overrider 2000 can also automatically maintain your Lintian overrides for you! It is simple, just do:

$ lintian -o <path/to/your/changes-file.changes>   \
      lintian-overrider --there-are-no-issues --source-dir <path/to/unpacked/source-tree>

Alioth is ready to take your git clone (or HTTP GET) request, so go order your copy now! </tv-shop-ad>